Typically, a streaming server encodes a two dimensional (2D) representation of a 360 degree video and communicates a portion of the encoded 2D representation to a device capable of rendering 360 degree video. Alternatively (or in addition), the streaming server encodes a three dimensional (3D) video representation of a 360 degree video. In one case encompassing two separate 2D views (one for each eye) or volumetric video is encoded and communicated, or another case where 2D or stereo video that includes depth information is encoded and communicated. The device then decodes the 2D (or 3D) representation, converts the decoded 2D representation to 360 degree video and renders the portion of the 360 degree video.